System and method for transcribing historical records into digitized text

ABSTRACT

A handwriting recognition system converts word images on documents, such as document images of historical records, into computer searchable text. Word images (snippets) on the document are located, and have multiple word features identified. For each word image, a word feature vector is created representing multiple word features. Based on the similarity of word features (e.g., the distance between feature vectors), similar words are grouped together in clusters, and a centroid that has features most representative of words in the cluster is selected. A digitized text word is selected for each cluster based on review of a centroid in the cluster, and is assigned to all words in that cluster and is used as computer searchable text for those word images where they appear in documents. An analyst may review clusters to permit refinement of the parameters used for grouping words in clusters, including the adjustment of weights and other factors used for determining the distance between feature vectors.

CROSS REFERENCES TO RELATED APPLICATIONS

This application claims the benefit of and is a non-provisional of U.S.Provisional Application Ser. No. 62/044,076 filed on Aug. 29, 2014,which is expressly incorporated by reference in its entirety for allpurposes as if fully set forth herein.

BACKGROUND OF THE INVENTION

Converting images of historical records into computer searchable textcan be challenging. Current systems often cannot identify individualwords within a digital image of historical records. Because words may bearranged in variable patterns and are often handwritten, accuratelyidentifying and converting individual words may require the efforts of aperson having experience in analysis of handwritten historical records.In some cases, the expert may have to review and manually enter text(manually transcribe words) corresponding to all or most of theindividual words in the historical record image.

Manually transcribing words in historical records is time-consuming andexpensive. Thus, historical records are often stored as digital imagedocuments (rather than as computer searchable text), and researchersneeding to search such documents for information are required to viewthem in order to find information.

There is thus arisen a need in the art for reducing the time and costfor producing searchable text versions of historical records.

BRIEF SUMMARY OF THE INVENTION

There is provided, in accordance with embodiments of the presentinvention, a network/system and method for creating a document withcomputer searchable text corresponding to word images in a handwrittendocument, such as a historical record. In embodiments, digital text isassigned to word images in the record based on the grouping of wordimages that likely represent the same handwritten word. A digital textword is assigned to word images that have been grouped together.

In one specific embodiment, a method for creating digitized text for arecord from an image of the record comprises obtaining one or moredigital images of a record, evaluating the digital images in order tolocate word images; for each located word image, identifying multipleword features of that word image; assigning, to one group (cluster) ofword images, designated ones of the multiple word images based on thedistance between the multiple word images, the distance representing thesimilarity of word features between at least two of the word images;selecting a representative word image (centroid) in the one group ofword images; selecting a digitized text for the representative wordimage; and assigning the selected digitized text to each of the wordimages in the one group of word images.

A more complete understanding of the present invention may be derived byreferring to the detailed description of the invention and to theclaims, when considered in connection with the Figures.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a general block diagram illustrating the general components ofa handwriting recognition system according to one embodiment.

FIG. 2 is a flow diagram illustrating the overall operation of thehandwriting recognition system seen in FIG. 1.

FIGS. 3A-O illustrate word features identified and analyzed by thehandwriting recognition system seen in FIG. 1.

FIG. 4 is a block diagram illustrating the functional components of theword feature extractor system seen in FIG. 1

FIG. 5 is a simplified representation of a feature vector for a wordimage (snippet).

FIG. 6 is a flow diagram illustrating a process for creating wordclusters.

FIG. 7 is a graphical representation of the calculation of distancebetween two snippets.

FIG. 8 is a flow diagram illustrating the calculation of distancebetween different types of word feature elements of two snippets.

FIG. 9 is a simplified representation of a matrix illustrating thecalculation of the distance between corresponding word features of twosnippets, using dynamic time warping (DTW).

FIG. 10 is a simplified graphical representation of several differentword clusters.

FIG. 11 is a graphical representation of a single word cluster,illustrating the arrangement of words or snippets in several differentregions of the word cluster.

FIG. 12 is a flow diagram illustrating the review, by an analyst, of theinnermost region of the word cluster seen in FIG. 11.

FIG. 13 is a flow diagram illustrating the review, by an analyst, of theintermediate and outermost regions of the word cluster seen in FIG. 11.

FIG. 14 is a block diagram illustrating an exemplary computer systemupon which embodiments of the present invention may be implemented.

DETAILED DESCRIPTION OF THE INVENTION

There are various embodiments and configurations for implementing thepresent invention. Generally, embodiments provide systems and methodsfor converting digital images of handwritten documents into computersearchable text.

In described embodiments, features of individual word images appearingon records, such as a historical record created in handwritten form andstored as a digital image, are identified. The word images are thenassigned to a group or cluster of word images based on common wordfeatures, with the word images in each cluster having a minimum orthreshold number of similar common word features. A selected digitaltext word (e.g., as entered by a human handwriting analyst) is assignedto each cluster, and the same digital text word is associated with theother word images in that cluster (and the record/document in which theword image appears), for purposes of making the record searchable fortext.

In one embodiment, the word images in a cluster are assigned a singledigital text word by selecting a representative word image in a cluster.The representative word may be a “centroid,” i.e., a word whose wordfeatures are most representative of the word features of all words inthe cluster. For example, the mean or average for each word feature ofall words in a cluster may be computed, and the centroid may be thesingle word image in the cluster whose word feature values are closestto the mean or average value of word features in the cluster. In someembodiments, certain word features may be preferred or weighted moreheavily than others for purposes of selecting the centroid.

Referring now to FIG. 1, the general components of a handwritingrecognition system 100 according to one embodiment of the invention areillustrated. The system 100 receives record/document images, i.e.,digital files representing images of historical records. As an exampleonly, historical records and other documents 102 may be scanned at anoptical scanner 104 that provides digital images to the system 100. Atits output, the system 100 assigns a digital text word to each (or most)of the individual word images on the received document images. Forexample, for each document image, the output of the system 100 providesdata representing each recognized handwritten word appearing on thedocument image (and, if appropriate, its location in the document), thatcan be stored and subsequently searched (as computer searchable text)for a person searching historical records for certain information, suchas genealogical information.

The functions of various components and subsystems of the system 100will be described in greater detail later. Overall, and as illustratedin FIG. 1, the system 100 includes a word locator module/system 110 forlocating individual word images (sometimes referred to herein as word“snippets”) on each document image. The individual word images are thenprovided to a word feature extractor module/system 120 that identifiesindividual features of each word image. The word feature extractormodule 120 provides at its output a feature vector for each word image,with each feature vector having values that reflect the characteristicsof various word features identified at the word feature extractor module120. The word feature vectors corresponding to word images are providedto a word classification/clustering system 140 which assigns eachindividual word image to a word group or cluster having an assignedcluster ID. Each word group or cluster is provided to a word clusterlabeling and refinement module/system 150 which assigns a digital textword to that cluster, based on review by a person, such as an expert inhandwriting analysis. The output of the word cluster labeling andrefinement system represents the digitized text assigned to each wordimage on each document image, as mentioned earlier.

It should be noted that the present invention is primarily intended tobe used with historical records that have handwritten information,especially in circumstances where the records may have degraded overtime. However, in broader aspects of the invention, features of theinvention may find use with more modern documents, including documentswhere information appears in printed form. This is illustrated by dashedlines in FIG. 1, where certain scanned documents (such asprinted-character documents), may be provided to a word/characterrecognition system 160, using well-known optical character recognition.In cases where optical character recognition is able to identify wordsin the documents, the identified word images may be provided at theoutput of the word recognizer module 160. In cases where some or all ofthe words in such documents are not identified, those documents orportions thereof maybe provided to the system 100 so that unidentifiedword images may be analyzed at the word feature extractor module 120,clustered at the word classification/clustering system 140, and thenhave text words assigned to those word images at the word clusterlabeling module 150. Thus, this last mentioned functionality of thesystem 100 may be used as an enhancement or substitution for opticalcharacter recognition systems in circumstances where the condition of adocument may not lend itself to accurate word or character recognitionand identification.

The overall operation of the system 100 will now be described in greaterdetail with reference FIG. 2. At step 210 a digital image of a record isobtained. The record may represent any one of many assembled historicalrecords containing genealogical information, such as birth records,census records, death records, marriage licenses, and other sources ofinformation pertaining to people. Printed or tangible records may bescanned by an optical scanner in order to obtain digital images, asmentioned earlier. Alternatively, the image may be provided from asource that has previously created images of historical records, such asa governmental or other entity that has previously scanned records foruse by researchers.

At step 212 the digital image is analyzed at word locator module 110 tolocate words (snippets) in the record. Various systems are currentlyavailable for locating word images in a document, such as documentlayout analysis functions used in word recognition systems availablefrom Kofax Limited, Irvine Calif., a subsidiary of LexmarkInternational, Inc., and from Abbyy USA, Milipits, Calif. Systems forlocating word images are also described in U.S. Pat. No. 6,249,604,entitled “METHOD FOR DETERMINING BOUNDARIES OF WORDS IN TEXT,” issued onJun. 19, 2001 to Huttenlocher et al., U.S. Pat. No. 6,393,395, entitled“Handwriting And Speech Recognizer Using Neural Network With SeparateStart And Continuation Output Scores,” issued on May 21, 2002 to Guha etal., and U.S. Pat. No. 5,442,715, entitled “METHOD AND APPARATUS FORCURSIVE SCRIPT RECOGNITION,” issued on Aug. 15, 1995 to Gaborski et al,each of which is hereby incorporated by reference.

In an alternative embodiment, a method known as “gridlining” may be usedto locate words. Gridlining may include, among other things, page layoutanalysis, table detection, field detection, and blank cell detection,and is described in U.S. application Ser. No. 13/601,111, entitledAutomated Field Position Linking of Indexed Data To Digital Images,filed Aug. 31, 2012, by Jack Reese et al., which is also herebyincorporated by reference. In yet another alternative embodiment,document layout analysis for locating word images could also beaccomplished manually by a person viewing document images on a computerscreen and selecting/marking individual words on the document (e.g., byplacing a graphical box or rectangle around each located word), andproviding marked individual word images to the word feature extractormodule 120.

At step 214, features of the located words are extracted and identifiedby the word feature extractor module 120. In one embodiment, wordfeatures extracted by the word feature extractor module 120 may includeTop Line Profile, Bottom Line Profile, Left Line Profile, Right LineProfile, Vertical Projection Profile, Horizontal Projection Profile,Local Maxima/Peaks, Local Minima/Valleys, Watershed Cup Areas, WatershedCap Areas, Loops-Holes, Intersections-Crossings, Convex Hull, StrokeOrientation/Slant, and Word Aspect Ratio. These features and the mannerof detecting and characterizing them in the form of a feature vector,will be described in greater detail below.

At step 216, words having similar word features are grouped or clusteredtogether at the word classification/clustering module 140. Briefly, andas mentioned earlier, each word image and its individual features arecharacterized in the form of a feature vector which has valuesrepresenting various features of the handwritten word in the image. Wordimages that have similar features are grouped together in a cluster.Algorithms will assign words to clusters, based on the similarity offeature vectors. The algorithms used will also cluster words even thoughthere may be variations in handwriting due to multipleauthors/enumerators that have created the original historical record.Word images in the same cluster have the same Cluster ID, also to bedescribed in greater detail later. As will also be described later, tominimize the “training” of algorithms which might otherwise requiremultiple iterations of development and feedback, a handwritinganalyst/expert or other human operator may subsequently examine arepresentative sample of words in each cluster to determine if theclustered words are in fact the same word, and change the weighting offeatures used to create each feature vector or make other changes toclustering algorithms to improve clustering accuracy.

At step 218, one word in each cluster is selected as a “centroid” by theword classification system 140, with the centroid having the mostrepresentative features of words (e.g., having features closest to theoverall mean or average of word features) in that cluster (other wordsin the cluster, while similar, may have fewer common features as onemoves away from the position of the centroid). The centroid in eachcluster is presented to a handwriting expert. At step 220, thehandwriting expert enters or keys the word (corresponding to thecentroid) as text at the word cluster labeling and refinement system150, and at step 222 that same word text is applied or assigned by thesystem 150 to all the words in that cluster.

At step 224, after clustered words have text associated with them, adigitized record with searchable computer-readable text is provided. Thedigitized record has each of its word images converted to digital textbased on the cluster to which each word image has been assigned (fromthe foregoing process). The digitized record may be used thereafter forsearching the historical records (as computer searchable text).

FIGS. 3A-3O illustrate an embodiment in which fifteen different wordfeatures are used by the word feature extractor system 120. For purposesof describing the word features and FIGS. 3A-3O, it is assumed that thedocuments on which word images appear have been provided as or convertedinto a binary image (binarized) to have only black and white colors orpixels, with black representing the foreground or “ink” (e.g.,handwriting), and white representing the background (e.g., the originalsheet of paper) on which the handwriting was placed. Well knownalgorithms such as Otsu binarization which maximize between groupvariance while minimizing within group variance are effective atbinarizing images, and in this case, image snippets. In some cases, wellknown binning can be used to reduce the number of pixels that need to beanalyzed. Further, each word image or snippet is cropped, so that word“signals” around the word that are weak may be removed and thereby leavea pre-established, clean, white space margin around the word.

Top Line Profile

A word profile may be thought of as similar to the familiar face profilewhich shows the outline of a face—from a profile view showing the neck,chin, mouth, nose, forehead, etc. The face profile is a condensedrepresentation which depicts some notable characteristics of the face.Similarly, a word profile depicts some notable profile attributes of theword. A top line profile represents the outline of the top-most inkpixels that make up the word, as illustrated in FIG. 3A. In other words,the word image is segmented into a plurality of vertical lines orcolumns (e.g., from left to right), with each column starting from thetop margin, and with the lowest point in each column defined by thetop-most pixel of the snippet in that column. When aggregated, thesignals or data outline represent the top-most pixels of the word (asrepresented in FIG. 3A by dotted lines).

Bottom Line Profile

A bottom line profile, as illustrated in FIG. 3B, is similar to the topline profile, but rather than an outline of top-most ink pixels, thebottom line profile (represented by dotted lines in FIG. 3B) representsan outline of bottom-most pixels that make up the word.

Left Line Profile

A left line profile, as illustrated in FIG. 3C, is similar to the topline and bottom line profile, but rather than an outline of the top-mostpixels or button-most pixels, a left line profile (represented by dottedlines in FIG. 3C) represents an outline of the left-most ink pixels thatmake up the word.

Right Line Profile

A right line profile, as illustrated in FIG. 3D, is similar to a leftline profile, but rather than using left-most ink pixels, the right lineprofile (dotted lines in FIG. 3D) represents an outline of theright-most pixels that make up the word.

Vertical Projection Profile

Projection profiles are similar to the top, bottom, left and rightprofiles, but are calculated by projecting lines through the wordsnippet space, and summing the number of ink pixels along thelines—essentially producing a histogram of the count of ink pixels alongthe projected lines through the words. Thus, a vertical projectionprofile, as illustrated in FIG. 3E, represents a histogram of ink pixelsin the word along vertically projected lines through each column of theword snippet. The resulting data/waveform signal illustrated in FIG. 3Erepresents a vertical projection as generated by projecting lines fromthe top margin of the snippet space.

Horizontal Projection Profile

A horizontal projection profile, as illustrated in FIG. 3F, is similarto the vertical projection profile, but represents a histogram of inkpixels in the word along horizontally projected lines through each rowof the word snippet. The resulting data/waveform signal illustrated inFIG. 3F represents a horizontal projection as generated by projectinglines from the right margin in the snippet space.

Local Maxima/Peaks

Local maxima/peaks represent the position and magnitude of the localmaxima (peaks) along the profiles described above. Peak features arecalculated for each of profiles described above (top, bottom, left andright profiles, and horizontal and vertical projection profiles). Anexample of local maxima/peaks calculated for a top profile isillustrated FIG. 3G. The identified peaks in FIG. 3G are designated bydotted circles.

Local Minima/Valleys

Local minima/valleys represent the position and magnitude of the localminima (valleys) along the profiles described above. Valley features arecalculated for each of the profiles described above (top, bottom, leftand right profiles, and horizontal and vertical projection profiles). Anexample of a minima/valleys calculated for a bottom line profile isillustrated in FIG. 3H. The identified valleys in FIG. 3H are designatedby dotted circles.

Watershed Cup Areas

Watershed cup features (as well as watershed cap features, to bedescribed shortly) are a corollary to the peaks and valleys describedabove and provide additional insight into word feature attributes.Watershed cups are formed between adjacent pairs of peaks along theprofiles. A watershed cup represents the watershed area between pairs ofadjacent peaks. Each Watershed cup feature is represented in twoparameters—the area of the cup and the median/center point within thecup. Watershed cups are associated with the areas above the profilebetween the neighboring local maxima/peak pairs. An example of a wordand its watershed cups areas are illustrated in FIG. 3I, with thewatershed cup areas shown with cross hatching.

Watershed Cap Areas

A watershed cap represents the established area between pairs ofadjacent valleys. A Watershed cap feature is represented in twoparameters—the area of the cap and the median/center point within thecap. Watershed caps are associated with the areas below the profilebetween the neighboring local minima/valley pairs. An example of a wordand its watershed cap areas are illustrated in FIG. 3J, with thewatershed cap areas shown with cross hatching.

Loops-Holes

Loops or holes in the word snippet correspond to loops or holes detectedin the binarized representation of the word snippet ink pixels. Loops(holes), therefore, are “background” (white) pixels entirely containedwithin, or surrounded by “ink” (black) pixels. Loop or hole features arerepresented as a list of parameter pairs including the area (e.g.,number of pixels), and the (x,y) coordinate position of the loop/holecenter point. An example of a word and its loops/holes are illustratedin FIG. 3K, with the loops-holes designated by dotted lines.

Intersections-Crossings

Intersections or crossings (e.g., a crossed “t”, or a number “8,” wherethe ink-stroke crosses over itself), are detected by well-known imageprocessing functions (e.g., morphological operators and filters)designed to detect pixel patterns representing such crossings. Thesecrossings are represented as a list of (x,y) coordinates of thecrossings. An example of a word and its intersections/crossings areillustrated in FIG. 3L, with the intersections/crossings designated bydotted lines.

Convex Hull

The Convex Hull is the polygonal convex hull that bounds the wordsnippet ink pixels, captured as the (x, y) point pairs that describe thebounding convex hull, and is illustrated for an exemplary word in FIG.3M. The bounding convex whole is designated by dotted lines in FIG. 3M.

Stroke Orientation/Slant

The predominant stroke orientation or slant of the characters in theword snippet is detected based on gradient orientation of thelongest/largest stroke components of the word snippet ink pixels. Theorientation (slant) feature is represented in a single parameter for theentire word snippet as the angle of orientation in degrees (e.g.,average value of longest stroke components). Because stroke slantvarious widely from person to person, this feature may be weighted lessthan many other features. The stroke orientation/slants (for which anaverage is computed) for an exemplary word are illustrated by dottedlines in FIG. 3N.

Word Aspect Ratio

The word aspect ratio is simply the x-y aspect ratio of the width-heightof the bounding rectangle that encloses the word snippet ink pixels. Theword aspect ratio for an exemplary word is illustrated in FIG. 3O, withthe bounding rectangle designated by dotted lines.

The following is Table I illustrates the values that are captured by theword feature extractor module 124 each of the features illustrated inFIGS. 3A-3O. As should be appreciated, for some word features theresulting data can take the form of a single array or set of elementsvalues, whereas the more complex features may take the form of multipledifferent types of elements and corresponding values for each. As willbe described later, the more complex features may be simplified by usinga Euclidean distance or calculation to create a single per featurevalue, which is then used in DTW algorithms for determining distance(similarities) between word images and thereby identify word images thatlikely represent the same word.

TABLE I Feature Nature of Values Top Line Profile For each top lineprofile pixel x (horizontal) position, a single value representing thecorresponding y location (vertical height) on the top line profile.Bottom Line Profile For each bottom line profile pixel x position, asingle value representing the corresponding y location (vertical height)on the bottom line profile position. Left Line Profile For each leftline profile pixel position, a single value representing thecorresponding x location (horizontal location) on the left line profileposition. Right Line Profile For each right line profile pixel position,a single value representing the corresponding X location (horizontallocation) on the right line profile position. Vertical Projection Foreach vertical projection line (column) in Profile the verticalprojection profile, a single value representing the magnitude of thevertical profile at that line. Horizontal Projection For each horizontalprojection line (row) in the Profile horizontal projection profile, asingle value representing the magnitude of the horizontal profile atthat line. Local Maxima/Peaks For each of the local maxima/peaks in theword, the x-y coordinates of that peak. Local Minima/Valleys For each ofthe local minima or valleys in the word, x-y coordinates of that valley.Watershed Cup Area For each watershed cup area in the word, the x- ycoordinates of the median/center point of the cup and the area or numberof pixels within the cup. Watershed Cap Area For each watershed cap areain the word, the x- y coordinates of the median/center point of the cap,and the area or number of pixels within the cap. Loops-Holes For eachloop or hole, the x-y coordinates of the median/center point of the loopor hole, and the area or number of pixels within the loop or holeIntersections-Crossings For each intersection, the x-y coordinates ofthat intersection. Convex Hull The x-y coordinates of each of the set ofpoints that define the convex hull, starting at the upper left andproceeding in clockwise order around the convex hull. StrokeOrientation/Slant The average orientation in degrees or radians of thedominant strokes detected across the word. Word Aspect Ratio For acropped word (e.g., around which a bounding rectangle or box islocated), the horizontal length or width (number of pixels) divided bythe vertical height (number of pixels)

Turning to FIG. 4, there is illustrated the overall operation of thefeature extractor module/subsystem 120 (FIG. 1). As seen, a selectedindividual digital word image or snippet 410 is provided to thesubsystem 120, with the selected word 410 separately provided to each ofa plurality of feature extractors 420 that each analyze the providedsnippet for a specific feature based on locations of the defining pixelsin the snippet (as described above in connection with FIGS. 3A-3O andTable I). Thus, the feature extractors 420 are individually designatedFeature Extractor 1 through Feature Extractor n, where n represents thetotal number of features to be extracted for each word (e.g., fifteenfeatures in the described embodiment, as illustrated in FIGS. 3A-3O).For example, Feature Extractor 1 could represent the digital analysis ofthe binarized word image for its top line profile and identifying, foreach top line profile pixel x (horizontal) position, a single valuerepresenting the corresponding y location (vertical height) on the topline profile of the snippet. Feature Extractor 2 could represent thedigital analysis of the binarized word image, providing valuesrepresenting for each bottom line profile pixel x position, a singlevalue representing the corresponding y location (vertical height) on thebottom line profile position, and so forth for each of the other fifteenfeatures in the described embodiment. The values representing the 15features are then assembled into a single multi-element feature vector425. Each feature vector is assigned a feature vector ID (that can beused to identify both the feature vector and its correspondingsnippet/word image). In the example seen in FIG. 4, for the individualidentified snippet (“Alabama”), the feature vector is assigned a featurevector ID “FV 32607.” This particular identified feature vector can bestored, retrieved and processed to develop word clusters (each snippetin the cluster representing a handwritten form of the same word). Whilenot illustrated in FIG. 4, other characteristics of the snippet can alsobe included in the feature vector, such as an identifier for thedocument image from which the snippet was obtained, as well as thelocation of the snippet on that document image. In some cases, a featurevector may include a weight for each feature that can be used incalculating distance or similarity between snippets (some features arecounted or more heavily weighted than others), also to be describedlater.

As mentioned, and as will be more fully described later, each featurevector 425 is represented at a high level (such as in FIG. 4) as alinear array of elements, one element corresponding to each feature.However, each element of the feature vector may in itself comprise anarray of data elements. In some cases, the feature is represented by asingle parameter, such as in the case of stroke orientation/slant (i.e.,the average angle of the slant) and word aspect ratio (the value of thewidth/height ratio). In other cases the feature may be represented by asimple string or array of multi-parameter values, such as the profilefeatures that consists, for each in a string of corresponding locations(along a single axis or direction), a value for each location. In othercases, the array of data elements for each feature may bemulti-dimensional and more complex. For example, the peaks, valleys,cups, caps, loops-holes, intersections-crossings and convex hullfeatures may consist of an array of elements, with each element in thatarray having multiple values (e.g., x-coordinate, y-coordinate and areavalue). As will also be more fully described later, well known dynamictime warping (DTW) may be used in some embodiments to determine thedistance (sometimes referred to as the “cost”) between word features andwords in order to identify snippets representing the same handwrittenword, but in other embodiments, for more complex features involvingmultidimensional arrays, the feature values may be simplified prior todetermining distance between snippets, using a Euclidean technique.

Various elements of an exemplary feature vector are illustrated in FIG.5. As seen, some elements, such as element 505 representing the strokeorientation feature, have a single parameter value (“74°”). Someelements, such as element 510 representing a top line profile feature,have a string of single values (“0, 0, 15, 16 . . . 957”), representingthe vertical value at each pre-established horizontal line/location.Still other elements are more complex and multidimensional, such aselement 515 representing peaks and having two values (x-y coordinates)for all peaks in a word (“x5, y27; x12, y307; x400, y12; etc.”) andelement 520, representing loops-holes and having triple values (x-ycoordinates and area) for loops in a word (“x15, y27, a3072; x42, y30,a4808; x510, y472, a2014; etc.”).

Turning now to FIG. 6, there is illustrated a process for creating wordclusters and a centroid for each of those clusters, such processgenerally performed within the word classification/clustering system 140seen in FIG. 1. The process, at its most basic level, involves thecalculation of distance (sometimes referred to as “cost”) between “wordpairs,” i.e., between each word or snippet in a typically largecollection of words and every other word or snippet in that largecollection of words. The collection of words may each be presented tothe word classification/clustering system in the form of the featurevector (from word feature extractor system 120) corresponding to each ofthe two words, where the feature vector (as noted earlier) may includenot only the multiple features that have been extracted from the word(by word feature extractor 120) but also the location of the word (allas uniquely identified by the feature vector ID).

As seen in FIG. 6, the process may include identifying wordneighborhoods, step 610, in order to more efficiently manage the numberof words that are being compared or analyzed together to calculate adistance. In other words, since the number of words being analyzed istypically very large, the word classification/clustering system 140 mayfirst attempt to group together words that may be roughly related orsimilar (“neighbors”), so that words that are highly unlikely to ever beclustered together (because they would never be viewed as beingrepresented by the same text word) are not analyzed as word pairs. Step610 may be accomplished in a number of different possible ways, such asby looking at word features at a very high level and creatingneighborhoods of words, where words appearing to begin with the sameletter, words having roughly the same length, words captured fromsimilar types of documents and/or words having other very high levelsimilarities may be put into the same “neighborhood.” It should beappreciated, of course, that this is merely a step for making theclustering process more efficient, and that if sufficient processingcapability is present, all the words provided by the word featureextractor 120 could be considered in a single neighborhood and all thosewords would be considered together for purposes of finding distancesbetween word pairs.

At step 612, the distance between each word and every other word in itsneighborhood is calculated, based on the analysis/comparison of thefeature vectors of those two words. The result of this step is a valuerepresenting the overall distance between those words, and this valueand the manner of obtaining it will be described in greater detail belowin conjunction with FIGS. 7 and 8. After a distance is calculatedbetween every word pair at step 612, the word classification/clusteringsystem 140 identifies, for each word, a second word in the neighborhoodthat it is closest to, and those two words are deemed (at leastinitially) to be a cluster (step 614). It is possible, at this step,that for each word being considered, two other words may be closest andof the same distance, in which case the cluster may be viewed as havingthree words, but such circumstance would likely be rare, particularly ifthere are a large number of features that are being considered for eachword. At step 620, one of the words in the initial cluster is selectedas the centroid. In one embodiment, the centroid may be the word in thepair that is closest to the mean or average of feature values for thetwo words in the pair, Since that word is most recent representative ofthe cluster. However, depending on the preferences of the user of thesystem, in alternative embodiments a centroid could be selected at thispoint by creating a phantom word or snippet that has exactly the mean oraverage feature values and using such a phantom centroid throughout theprocess of FIG. 6 until the creation of the cluster has been completed.

Next, at step 626, the next closest word (by distance) to the centroidis added to the cluster. Since the cluster now includes an additionalword, the mean or average of feature values will most likely havechanged, and the centroid is updated, i.e., the word in the expandedcluster that is now closest to the mean or average of feature values isselected as the centroid (of course, in some cases, the updated centroidmay be the same centroid that was selected at step 620). The foregoingsteps are repeated in order to continue to build the cluster (at step626) and update the centroid (step 630) until the desired size of thecluster has been reached at step 640. There may be various ways ofdetermining when to stop the building of a cluster. For example, thesystem might be designed to have clusters with no more than a certainmaximum number of words, and when that maximum is reached at step 640,the clustering ceases for that cluster. Alternatively, since the processof FIG. 6 is being performed across a large collection of words, withmultiple clusters being simultaneously built, when the largest orsmallest of those clusters reaches a specified maximum size, theclustering process may be stopped. Words at that point that have notbeen placed in a cluster may be set aside and used later with adifferent collection of words to see if they can be clustered later withother word images. Alternatively, in some circumstances, theun-clustered words might be viewed as so unique that they will beassigned a text word separate and apart from any clustering process(such as by an handwriting expert individually looking at those wordimages).

As will be described in greater detail later in conjunction with FIG.11, in some embodiments a cluster may be constructed with multipleregions, such as three regions reflecting an innermost region or ring ofsnippets that would include the centroid, an intermediate region or ringof snippets further from the centroid and thus being less similar to thecentroid than the snippets in the innermost region, and an outermostregion or ring of snippets even further from the centroid and even lesssimilar to the centroid of the snippets in the innermost andintermediate regions. It should be appreciated that these regions couldbe established by different thresholds being reached at step 640. Forexample, a first threshold may establish the boundary of the innermostregion, a second threshold could establish the boundary between theinnermost and intermediate regions, and a third threshold (representingthe threshold at which the building of the cluster stops) wouldrepresent the boundary around the outermost region.

With the clustering stops, each created cluster represents a group ofwords that have a threshold or minimum similarity to each other (asdetermined by the distance between the centroid and the most distantwords/snippets in the cluster). It should be appreciated that inpractice a cluster could be small (just a few snippets/word images), butin many or most cases a cluster might be very large (thousands or moreof snippets).

Once the clustering process has been completed at step 644 for each ofthe clusters created in the word classification/clustering system 140, acluster ID is assigned to each of you who will cluster at step 650, andthe process ends.

FIG. 7 is a graphical illustration of the calculation of distancebetween a word pair (performed at step 612 in FIG. 6), in particular thedistance between a word or snippet 710 (“Arkansas”) and a word orsnippet 720 (“Alabama”). As discussed earlier, for purposes ofcalculating a distance, feature vectors 726 and 728 for the two words inquestion are provided by the word feature extractor system 122 to theword classification/clustering subsystem 140. These feature vectors(labeled “Feature Vector A” and “Feature Vector B” in FIG. 7) areprovided to a feature distance calculator 732 within wordclassification/clustering system 140. The calculator 732 calculates adistance between the corresponding features of the feature vectors 726and 728, and provides at its output an overall distance between thewords as determined by aggregating the individual distances between thecorresponding features (feature vector elements) of the two featurevectors. For example, the distance provided at the output of calculator732 may be the distance between corresponding individual features thatare then summed, with the individual features weighted according tofeature weights provided by stored feature weighting values 740. Theweights may be determined in advance in order to make sure that theindividual features (and the numerical values that are generated) areappropriately considered when calculating the overall distance betweenthe two words. Weights may be determined in advanced based on pastexperiences or judgment of a handwriting expert. As mentioned earlier,in some embodiments the weights might be included in the feature vector,particularly in circumstances where weights may need to be based on thebroad category of words involved. For example, weights might be based onthe time period of the documents being analyzed and known writing stylesduring that time period (e.g., during certain historical periods,handwriting customary for certain kinds of legal documents mightminimize differences between capital and lowercase letters and sovertical projection profiles, peaks and watershed areas may be lessuseful in distinguishing words and might be weighted less). As anotherexample, for certain types of records, it may have been customary duringcertain historical periods to abbreviate words by truncating them inorder to conserve space, and so the word aspect ratio for words on thosedocuments might be weighted less. In these examples, the weightsinvolved might be included with all feature vectors for words taken fromthose documents. Further, weights applied to specific features may beused to scale those features to normalize measurements (e.g., strokeorientation/slant may be measured in degrees typically in excess of 100,whereas word aspect ratio will be typically be a fraction), andweighting compensates for each standard of measurement and its units sothat the standard of measurement for individual features itself does notinappropriately skew the results.

In other cases, the weights may be provided independently of the featurevectors, based on feedback received from an handwriting expert involvedin the creation of digital text (a person reviewing the results of theclustering and making adjustments to the manner in which words are beingclustered in order to correct cluster errors, to a described in moredetail later in conjunction with FIGS. 12 and 13).

As mentioned earlier in conjunction with FIG. 5, the nature of a featurevector itself (and differing numbers of dimensional values that may beused to represent each feature) provide some complexity to the featurevector. As such, different types of computations and algorithms may beused to calculate the distance for individual vector features within thefeature distance calculator 732 of FIG. 7. This is illustrated in theprocess of FIG. 8, which illustrates three different calculations ofdistances between corresponding feature elements of a word pair, basedon the dimensional nature of the feature value.

The first calculation illustrated in FIG. 8 is for single valuefeatures, such as word aspect ratio or stroke orientation/slant (see,e.g., feature element 505 and FIG. 5, representing strokeorientation/slant). The distance between the corresponding features oftwo words is calculated by simply determining the difference between thefeature values at step 810.

The second calculation illustrated in FIG. 8 is for features involving astring of element values, and the computation is more involved. Anexample of such a feature would be feature element 510 in FIG. 5,representing a top line profile (each value representing the verticalheight or top-most pixels of successive horizontal points along theword). The calculation of distance between the two words for thisparticular feature (this distance sometimes referred to as “cost”) iscalculated using well-known dynamic time warping (DTW) algorithms atsteps 820 and 822. Dynamic time warping compares sequences of values(such as feature values that vary across a data signal—e.g., aword—during a time dimension or the like) by shifting the correspondingpairs of values that are compared over the sequence.

The DTW technique is well known and is illustrated in simplified form inFIG. 9 as a matrix, where a string of values for a Word 1 (positions 0through 9, across the top of the matrix) are compared to a string ofvalues for a Word 2 (positions 0 through 9, across the left side of thematrix). In this simple example, the values being compared may be astring of top line profile values (2, 3, 2, 9, 12, 9, 2, 3, 2, 3) forWord 1 and a corresponding string of top line profile values (3, 2, 3,8, 13, 9, 3, 2, 3, 2) for Word 2, where the matrix values are firstpopulated by determining the difference between each value in Word 1 toevery value in Word 2 (step 820 in FIG. 8). Then at step 822, theoptimal or shortest distance path across the matrix is determined (path910) with the values across the path summed to determine the overalldistance between the two word features (in the simplified example inFIG. 9, the total distance between the corresponding word features ofWords 1 and 2 is 7).

It should be appreciated, that the various values shown in FIG. 9 aregreatly simplified and the example is provided simply for the purposesof illustrating the process as it might be used in more complex wordfeatures, to be discussed later. Far more detail concerning the natureof DTW calculations can be found in many publications, including, forexample, F. Itakura, “Minimum Prediction Residual Principle Applied toSpeech Recognition,” IEEE Transactions on Acoustic Speech & SignalProcessing, vol. ASSP-23, p. 67-72, February, 1975; and H. Sakoe and S.Chiba, “Dynamic Programming Algorithm Optimization for Spoken WordRecognition,” IEEE Transactions on Acoustic Speech & Signal Processing,vol. ASSP-26, No. 1, February, 1978), and in various prior patents, suchas aforementioned U.S. Pat. No. 6,393,395, entitled “HANDWRITING ANDSPEECH RECOGNIZER USING NEURAL NETWORK WITH SEPARATE START ANDCONTINUATION OUTPUT SCORES,” issued on May 21, 2002 to Guha et al.; U.S.Pat. No. 5,664,058, entitled “METHOD OF TRAINING A SPEAKER-DEPENDENTSPEECH RECOGNIZER WITH AUTOMATED SUPERVISION OF TRAINING SUFFICIENCY,”issued on Sep. 2, 1997 to Vysotsky; U.S. Pat. No. 4,918,733, entitled“DYNAMIC TIME WARPING USING A DIGITAL SIGNAL PROCESSOR,” issued on Apr.17, 1990 to Daughtery; and U.S. Pat. No. 4,88,243, entitled “DYNAMICTIME WARPING ARRANGEMENT,” issued on Dec. 11, 1984 to Brown et al., eachof which is hereby incorporated by reference.

The third calculation illustrated in FIG. 8 is for multi-dimensionalfeature value elements, such as feature elements 515 and 520 in FIG. 5representing, respectively, peaks having two values (x-y coordinates)and loops-holes having triple values (x-y coordinates and area). In thisthird calculation, a Euclidean distance calculation is first determinedfor each element value at step 830.

Thus, for x-y coordinates the Euclidean distance would be calculated as:

Distance=√{square root over (x ² +y ²)}

For x-y coordinates and area (a) the Euclidean distance would becalculated as:

Distance=√{square root over (x ² +y ² +a ²)}

At step 832, the Euclidean distance would then be used to populate theDTW algorithm and matrix (such as the matrix seen in FIG. 9, with theEuclidean distance value substituted for the single values shown forWord 1 and Word 2), and at step 834 the distance or cost between thosecorresponding word features of Word 1 Word 2 would be calculated usingthe DTW algorithm.

At step 836, the distance between corresponding features of the twowords would be summed together (with individual features weighted ornormalized, as discussed earlier) to provide the overall distancebetween the two words.

While described embodiments use dynamic time warping to determine costor distance between certain types feature values, other forms ofanalysis could be used for clustering and recognizing words, such asBayesian networks and neural networks (e.g., convoluted neural networks)

Turning now to FIG. 10, a simplified illustration of word clusters forseveral different word images is seen. It should be appreciated that inactual practice, both the number of clusters and the number of wordimages or snippets in each cluster is likely to be very large and,hence, much larger than as illustrated in FIG. 10. In FIG. 10, sevenword clusters are illustrated, identified as Cluster ID 100 (snippetsrepresenting word “Arkansas”), Cluster ID 110 (snippets representing theword “Arkansas,” but having word features sufficiently different fromthose in cluster ID 100, and thus in a separate cluster from Cluster ID100), Cluster ID 200 (snippets representing the word “Arizona”), ClusterID 300 (also snippets representing the word “Arizona,” but having wordfeatures sufficiently different from those in Cluster ID 200), ClusterID 400 (also snippets representing the word “Arizona,” but having wordfeatures sufficiently different from those in Cluster IDs 200 and 300),Cluster ID 500 (snippets representing the word “Alabama”), and clusterID 600 (also snippets representing the word “Alabama,” but having wordfeatures sufficiently different from those in cluster ID 500).

Also illustrated in FIG. 10 is a snippet 1010 (shown in dotted lines)representing the centroid for Cluster ID 100, snippet 1020 representingthe centroid for Cluster ID 110, a snippet 1030 representing thecentroid Cluster ID 200, a snippet 1040 representing the centroid forCluster ID 300, a snippet 1050 representing the centroid for Cluster ID400, a snippet 1060 representing the centroid for Cluster ID 500 and asnippet 1070 representing the centroid for Cluster ID 600. FIG. 10 alsoillustrates several errors in the clustering process, including snippets1080 and 1081 which have been erroneously grouped in Cluster ID 100, asnippet 1082 which has been erroneously clustered in Cluster ID 300, anda snippet 1083 which has been erroneously clustered in Cluster ID 500.The process for identifying and removing erroneous snippets will bedescribed below.

FIG. 11 is a graphical representation of a single word cluster 1100. Aswith other clusters, cluster 1100 represents word images that have beengrouped together as likely representing the same single word, which wordis represented by the designation “W” in FIG. 11. The cluster 1100 has acentroid 1110 (illustrated graphically in bold and at the center of thecluster 1100). The cluster 1100 is also graphically represented ashaving three regions defined by circular boundaries. The first region1120 represents the innermost part/region of the cluster (having wordsor snippets with word feature vectors closest in distance or similarityto the centroid and thus having the highest likelihood of beingcorrectly included in the cluster). The second region 1122 represents anintermediate part of the cluster (having words or snippets with wordfeature vectors that are close to the centroid, but not as close as thesnippets in region 1120) and thus have a likelihood (but not as high asthe snippets in region 1120) of being correctly included in the cluster.The third region 1124 represents the outermost part of the cluster thathas words or snippets that are loosely similar to the centroid. Snippetsin region 1124 are seen to include words W₁ (designated in FIG. 11 byarrow 1142), W₂ (designated in FIG. 11 by arrow 1144) and W₃ (designatedin FIG. 11 by arrow 1146) that have all been erroneously placed incluster 1100 (they are not the word W) and that can be removed using aprocess involving human interaction that will be described shortly.

As mentioned earlier, a handwriting analyst or expert is involved in theuse of clusters for assigning text words to word images or snippets. Inthe described embodiment, an analyst performs, in conjunction withvarious subsystems within the handwriting recognition system 100, atleast two functions: (1) keying or entering a digital text word that isrecognized when viewing the centroid (the digital text word is thenassociated with/assigned to all the word images or snippets in thecluster) and (2) examining samples of words in clusters to determine thescope of any errors and whether adjustments need to be made to thealgorithms involved in assigning word images to the clusters. Functionsperformed with the use of a handwriting analyst are illustrated in FIGS.12 and 13.

The entry of a digital text word for the centroid may be a relativelystraightforward process—the analyst has the centroid for a clusterpresented, e.g., at a computer screen associated with the handwritingrecognition system 100, recognizes (in a vast majority of cases) thecentroid word image as a specific handwritten word (e.g., based on theexperience/skill of the analyst), and then enters the recognized word(e.g., at a keyboard), with the word cluster labeling and refinementsystem 150 associating the entered word with every word image in thatcluster. However, as mentioned earlier, because of the wide variety ofhandwriting styles, there will invariably be a few word images in thecluster that have been placed there in error and the handwriting analystwill examine clusters in order to reduce the number of errors and refinethe algorithms that are being used to place word images in clusters.

For purposes of the present description, it is assumed that each clusterhas been divided into three regions (as illustrated in FIG. 11), andthat the process used for examining the cluster for errors in theinnermost region (the first region 1120, FIG. 11), where there is afairly high level of confidence about the grouping of the snippets inthat region, may be different than the process used for examining thecluster for errors in the intermediate and outermost regions of thecluster (the second region 1122 and the third region 1124, FIG. 11).

FIG. 12 illustrates a process that may be used by an analyst inconjunction with word cluster labeling and refinement system 150 forexamining word images in the innermost region (first region 1120). It isassumed before the examination of any regions that the analyst hasalready keyed in or entered a word for the centroid, as illustrated bystep 1210 in FIG. 12. The analyst then reviews a sample of snippets inthe first region at step 1214. The review of the snippets may take theform of representative snippets being sequentially displayed to theanalyst at a computer screen, with the analysts marking any snippet thatis not the same word as represented by the centroid. Because of the highlevel of confidence in correctness of words in first innermost region,the sample of snippets reviewed at step 1214 may be relatively small,e.g., less than 10% of the snippets in the first region.

At step 1216 the system 100 (in particular, word cluster labeling andrefinement subsystem 150) determines whether a high percentage of thesample snippets are correct based on input from the analyst. Thispercentage will normally be high (e.g., 99%) because, absent afundamental flaw in the selection of word images for the cluster, thereshould be very few if any errors. If a high percentage of the samplesnippets are correct at step 1216, then the cluster is maintained, step1218. As part of maintaining the cluster, analysts may review theindividual erroneous snippets to see if there are obvious reasons fortheir presence in the cluster. For example, there may be variations inthe spelling of the same word, or two different words may have a singleword feature that has similar characteristics as observed by theanalyst, and the analysts can determine, for example, if that particularword feature should receive less weight when assembling the word featurevector and calculating distance. In some cases, the analyst maydetermine the errors are so insignificant that they will not interferewith most research to be done with the documents, and may ignore anyerrors at steps 1214 and 1216.

If a high percentage of the sample snippets are not correct at step1216, then the system 100 will normally provide a larger sample ofsnippets (higher than the 10% initially reviewed at step 1214), at step1224. If a high percentage of the larger sample is correct, step 1230,then the cluster is maintained (step 1218), with the analyst having theoption of reviewing any of the small number found to be incorrect atstep 1230 and making appropriate corrections, as mentioned in connectionwith erroneous snippets determined at step 1214 and 1216. However if ahigh percentage of the larger sample of snippets are not correct step1230, then the cluster is marked as suspicious (step 1234) and is notused for purposes of assigning an identified text word to the cluster.Such a condition may be the result of basic flaws in the algorithm usedfor clustering the snippets, and may require a more thorough review bythe analyst to correct the problem.

FIG. 13 illustrates a process that may be used by an analyst inconjunction with system 100 for examining word images in theintermediate and outermost regions (regions 1122 and 1124) of the wordcluster 1100 illustrated in FIG. 11. It is assumed before examination ofthese regions, as was assumed in connection with the process illustratedin FIG. 12, that the analyst has already keyed in or entered a word forthe centroid, as illustrated by step 1310 in FIG. 13. The analyst thenreviews a sample of snippets in regions 1122 or 1124, step 1312. Thesample of snippets at step 1312 would be expected to be larger than thesample of the innermost region at step 1214, FIG. 12. It should beappreciated that, because the confidence level about the grouping ofsnippets in the intermediate and outermost regions may be different, thesample of snippets reviewed at step 1312 for those two regions may bedifferent. For example, in one embodiment, there may be a medium levelof confidence in the correctness of words in the intermediate region,and the sample of snippets may be, for example, 25% of the snippets inthat region. There may be a low level of confidence in the correctnessof words in the outermost region 1124, and a sample of snippets may be,for example, in the range of 50 to 80% for that region. At step 1316,the word cluster labeling and refinement system 150 determines whetherthe established threshold percentage of sample snippets is correct basedon input from the analyst. It is expected that the threshold percentagewill be relatively high (e.g., 90%) since a high number would beexpected in order to keep those regions in the cluster. If the thresholdpercentage is in fact correct, those regions are maintained as part ofthe cluster, at step 1318. As part of maintaining the cluster, and asdiscussed in connection with the process of FIG. 12, the analysts mayreview the individual or erroneous snippets to see if there are obviousreasons for their presence in the cluster and with the analyst makingadjustments, such as weights used in calculating distances and makingcorrections to erroneous snippets labels, similar to the mannerdescribed in conjunction with FIG. 12.

If the threshold percentage is not correct at step 1316, then a largersample of snippets may be reviewed at step 1324. If the correct snippetsare within the threshold percentage, step 1330, then those regions aremaintained as part of the cluster at step 1318. If the larger sample ofsnippets falls outside the threshold percentage at step 1330, then theregion may be removed from the cluster at step 1334 and those snippetsor words may be returned for further processing, step 1340, to determineif they should be included in a different cluster (such as by repeatingthe cluster building process for those snippets, as described earlier inconjunction with FIG. 6).

FIG. 14 is a block diagram illustrating an exemplary computer systemupon which embodiments of the present invention may be implemented. Thisexample illustrates a computer system 1400 such as may be used, inwhole, in part, or with various modifications, to provide the functionsof the system 100, including the word locator system 110, word featureextractor system 120, word classification/clustering system 140, andword cluster labeling and refinement module/system 150, as well as othercomponents and functions of the invention described herein.

The computer system 1400 is shown comprising hardware elements that canbe electrically coupled or otherwise in communication via a bus 1405.The hardware elements can include one or more processors 1410,including, without limitation, one or more general-purpose processorsand/or one or more special-purpose processors (such as digital signalprocessing chips, graphics acceleration chips, and/or the like); one ormore input devices 1415, which can include, without limitation, a mouse,a keyboard and/or the like; and one or more output devices 1420, whichcan include, without limitation, a display device, a printer and/or thelike.

The computer system 1400 may further include one or more storage devices1425, which can comprise, without limitation, local and/or networkaccessible storage or memory systems having computer or machine readablemedia. Common forms of physical and/or tangible computer readable mediainclude, as examples, a floppy disk, a flexible disk, hard disk,magnetic tape, or any other magnetic medium, an optical medium (such asCD-ROM), punchcards, papertape, any other physical medium with patternsof holes, a random access memory (RAM), a read only memory (ROM) whichcan be programmable or flash-updateable or the like, and any othermemory chip, cartridge, or medium from which a computer can read data,instructions and/or code. In many embodiments, the computer system 1400will further comprise a working memory 1430, which could include (but isnot limited to) a RAM or ROM device, as described above.

The computer system 1400 also may further include a communicationssubsystem 1435, such as (without limitation) a modem, a network card(wireless or wired), an infra-red communication device, or a wirelesscommunication device and/or chipset, such as a Bluetooth® device, an802.11 device, a WiFi device, a WiMax device, a near fieldcommunications (NFC) device, cellular communication facilities, etc. Thecommunications subsystem 1435 may permit data to be exchanged with anetwork, and/or any other devices described herein. Transmission mediaused by communications subsystem 1435 (and the bus 1405) may includecopper wire, coaxial cables and fiber optics. Hence, transmission mediacan also take the form of waves (including, without limitation radio,acoustic and/or light waves, such as those generated during radio-waveand infra-red data communications).

The computer system 1400 can also comprise software elements,illustrated within the working memory 1430, including an operatingsystem 1440 and/or other code, such as one or more application programs1445, which may be designed to implement, as an example, the processesseen in FIGS. 2, 6, 8, 12 and 13.

As an example, one or more methods discussed earlier might beimplemented as code and/or instructions executable by a computer (and/ora processor within a computer). In some cases, a set of theseinstructions and/or code might be stored on a computer readable storagemedium that is part of the system 1400, such as the storage device(s)1425. In other embodiments, the storage medium might be separate from acomputer system (e.g., a removable medium, such as a compact disc,etc.), and/or provided in an installation package with theinstructions/code stored thereon. These instructions might take the formof code which is executable by the computer system 1400 and/or mighttake the form of source and/or installable code, which is compiledand/or installed on the computer system 1400 (e.g., using any of avariety of generally available compilers, installation programs,compression/decompression utilities, etc.). The communications subsystem1435 (and/or components thereof) generally will receive the signals(and/or the data, instructions, etc., carried by the signals), and thebus 1405 then might carry those signals to the working memory 1430, fromwhich the processor(s) 1405 retrieves and executes the instructions. Theinstructions received by the working memory 1430 may optionally bestored on storage device 1425 either before or after execution by theprocessor(s) 1410.

While various methods and processes described herein may be describedwith respect to particular structural and/or functional components forease of description, methods of the invention are not limited to anyparticular structural and/or functional architecture but instead can beimplemented on any suitable hardware, firmware, and/or softwareconfiguration. Similarly, while various functionalities are ascribed tocertain individual system components, unless the context dictatesotherwise, this functionality can be distributed or combined amongvarious other system components in accordance with different embodimentsof the invention. As one example, the system 100 may be implemented by asingle system having one or more storage device and processing elements.As another example, the system 100 may be implemented by plural systems,with their respective functions distributed across different systemseither in one location or across a plurality of linked locations.

Moreover, while the various flows and processes described herein aredescribed in a particular order for ease of description, unless thecontext dictates otherwise, various procedures may be reordered, added,and/or omitted in accordance with various embodiments of the invention.Moreover, the procedures described with respect to one method or processmay be incorporated within other described methods or processes;likewise, system components described according to a particularstructural architecture and/or with respect to one system may beorganized in alternative structural architectures and/or incorporatedwithin other described systems. Hence, while various embodiments may bedescribed with (or without) certain features for ease of description andto illustrate exemplary features, the various components and/or featuresdescribed herein with respect to a particular embodiment can besubstituted, added, and/or subtracted to provide other embodiments,unless the context dictates otherwise. Consequently, although theinvention has been described with respect to exemplary embodiments, itwill be appreciated that the invention is intended to cover allmodifications and equivalents within the scope of the following claims.

What is claimed is:
 1. A method for creating digitized text for a recordfrom an image of the record, comprising: obtaining one or more digitalimages of a record; evaluating the digital images in order to locateeach of multiple word images; for each located word image, identifyingmultiple word features of that word image; assigning, to one group ofword images, designated ones of the multiple word images based on thedistance between the multiple word images, the distance representing thesimilarity of word features between at least two of the word images;selecting a representative word image in the one group of word images;selecting digitized text for the representative word image; andassigning the selected digitized text to each of the word images in theone group of word images.
 2. The method of claim 1, wherein the recordis a historical record having handwritten words, and wherein themultiple word images are each an image of one of the handwritten words.3. The method of claim 1, wherein the word images each havecorresponding word features, and wherein the method further comprises:assigning a value to each of the word features; and calculating adistance between the at least two of the word images, based on thedifference between values of corresponding word features of those twowords.
 4. The method of claim 1, further comprising: calculating a meanof values for word features of the multiple word images assigned to theone group of word images, wherein the step of selecting a representativeword image comprises selecting a word image in the one group of wordimages that is closest to the mean.
 5. The method of claim 1, whereinthe step of selecting digitized text for the representative word imagecomprises: displaying the representative word image to a handwritinganalyst; and receiving the selected digitized text from the handwritinganalyst based on the displayed representative word.